home *** CD-ROM | disk | FTP | other *** search
Text File | 1994-10-01 | 3.7 KB | 95 lines | [TEXT/MMCC] |
- Enclosed in this folder are two programs based on a token generating engine.
- Tokenizer was a test program that I created to see how hard it would be to create
- a language sensitive editor. It looks very promising since most of the work
- is being done by Apple's Script Manager. Drop Pretty print is the first program
- based on the Tokenizer. Its a Simple Drag and drop source code formatted source
- code printer.
-
- What Tokenizer does right now.
-
- It will tokenize a file it recognizes and displays the text formatted into a
- fixed 640x480 window. If the file is less than the number of lines displayed it
- displays the text and then exits (not very useful I know). Once you open a file
- you have to hold down the mouse button to scroll through the file. Again not
- very useful and its the only thing you can do until its done displaying the
- file. I haven't added a lot of user interaction because the first versions will
- be drag and drop applications. The user interface will be added when the editor
- starts taking shape. So don't drop long files onto it unless you like to hold
- down the mouse button a lot . :-)
-
-
- The point of this exercise is to see how it works and to report things that it
- doesn't recognize. Also I would like people to try and expand the language
- definitions to see if it can parse other languages. When I first posted about
- this program it only recognizes .c and .p files. In less than one minute I added
- .r and .h files support. Chris Thomas added c++ support.
-
- To add a language there are two custom resources. LANG and COMM. Lang describes
- the suffix and the keywords in that file (language). The COMM is the comment
- delimiters for the language. the two work as indexed pairs. LANG is self
- explanatory I think. To add C++ files just duplicate the C resource add or delete
- any keywords for the language and duplicate the C COMM resource since c++
- comments are probably the same. To change the comments you will need the Script
- manager documentation that's in IM v6 or IM:Text. It uses the tokenXXX constants
- for character definitions.
-
- suffixes are limited to 32 characters
-
- Drop Pretty Print
-
- Drop pretty Print is a simple program based on the Tokenizer code that will
- send your source code formatted to the current printer. The header and footer
- are customizable via ResEdit. The header has four variables to describe the
- filename, modified date, modified time and the creator.
-
- &f = filename
- &d = date
- &t = time
- &c = creator
-
- The footer has one variable called &p for the page number. The header and footer
- strings are in the STR# resources.
-
- The font and sized used are stored in the 'STR ' resources.
-
-
- What I hope it will become
-
- A language sensitive programmers editor. Since the program actually tokenizes the
- text and will probably store the text in tokenized form, I hope that the compiler
- will in some way support smart editors. If the compiler can accept tokens
- instead of raw text and know that its syntactically correct, we can probably
- improve the speed of edit-compile-link cycle even further.
-
-
- So please let me know what features you would like to see in each of the
- "versions"
-
- The resources for LANG and COMM are goofy and they have been changed in the next
- release.
-
-
- As to the source code.
-
-
- These isn't much there. the
- TokenResults:=IntlTokenize(tokenParam_TokenBlockPtr); function handles all of
- the work, its in Pascal, and I think Peter, Quinn and I are the only ones still
- using this language :-). If I clean up the code and make it palatable for general
- perusal or if I can't find time to work on the program I will release the
- source. Plus I couldn't release it until I can get it converted to MW Pascal. :-)
-
-
- Matthew Xavier Mora
-
- MXM Designs™
- 39075 Carmel Ct
- Fremont CA 94538
- Voice: (510) 795-1978
- Email: mxmora@unix.sri.com
-
- P.S. Yes I know the icons are really ugly. :-)
-
-
-
-